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The  Human  Factors  Technical  Area  of  the  Army  Research  Institute 
(ARI)  is  concerned  with  the  demands  of  the  future  battlefield  for 
increased  man-machine  complexity  to  acquire,  transmit,  process, 
disseminate,  and  utilize  information.  The  research  is  focused  on  the 
interface  problems  and  interactions  within  command  and  control  centers 
and  is  concerned  with  such  areas  as  topographic  products  and  procedures, 
tactical  symbology,  information  management,  user-oriented  systems,  staff 
operations  and  procedures,  and  sensor  systems  integration  and 
utilization. 

One  area  of  special  interest  is  that  of  human  factors  problems  in 
the  design  of  topographic  products  and  procedures.  This  publication 
evaluates  the  effectiveness  for  solving  military  map  problems  of  supple¬ 
menting  current,  standard  relief  formats  with  either  shaded  relief  or 
layer  tints.  Findings  can  aid  in  the  design  of  cost-effective  special 
purpose  Army  maps  and  in  the  objective  evaluation  of  cartographic 
products . 

Research  on  the  design  of  topographic  products  and  procedures  is 
conducted  as  an  in-house  effort  augmented  by  contracts  with  organiza¬ 
tions  selected  for  their  specialized  capabilities  and  facilities.  These 
efforts  are  responsive  to  the  general  requirements  of  Army  Project 
2Q763743A774  and  to  special  requirements  of  the  USA  Engineering 
Topographic  Laboratory.  Special  requirements  are  contained  in  Human 
Research  Need  79-230  "Topographic  Product  Design  and  Test  Methodology." 

This  research  effort  was  made  possible  through  the  assistance  of 
Alex  Pearson  of  the  USA  Engineering  Topographic  Laboratory  and  through 
the  cooperation  of  Combat  Developments  Experimentation  Command,  Fort 
Ord,  California. 
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A  COMPARISON  OF  ALTERNATE  FORMATS  FOR  THE  PORTRAYAL  OF  TERRAIN  RELIEF  ON 
MILITARY  MAPS 

BRIEF 


Requirement : 

To  evaluate  techniques  designed  to  enhance  the  display  of  terrain 
information  on  military  maps,  and  compare  the  effectiveness  of  these 
techniques  with  standard  U.S.  Army  contour  line  maps. 


Procedure : 

In  two  experiments,  the  standard  U.S.  Army  contour  line  maps  were 
compared  with  innovative  map  formats  consisting  of  contour  lines  supple¬ 
mented  with  (a)  relief  shading,  and  (b)  gradient  (layer)  tints.  The 
effectiveness  of  the  maps  was  evaluated  by  measuring  the  accuracy  and 
speed  with  which  topographic  information  was  extracted  from  the  map 
products.  A  field  trial  was  also  conducted  to  compare  the  performance 
criteria  used  in  Experiments  I  and  II  with  performance  on  map-use  tasks 
in  the  field,  i.e.,  finding  checkpoints  and  reporting  target  locations. 


Findings  : 

The  addition  of  layer  tints  to  maps  increases  the  speed  with  which 
information  can  be  extracted  from  maps  with  no  loss  in  accuracy. 

The  addition  of  relief  shading  increases  the  amount  of  time 
required  to  extract  the  topographic  information  needed  in  the  map-use 
tasks  of  most  Army  personnel,  although  relief  shading  has  some  value  for 
visualizing  terrain. 

Experienced  personnel  solve  map  tasks  more  accurately,  but  not 
necessarily  faster  than  less  experienced  personnel. 


Utilization  of  Findings: 

Layer  tints  are  a  valuable  supplement  on  special  purpose  maps  for 
fast  and  accurate  solutions  to  terrain  information  problems,  especially 
if  the  problems  involve  elevation.  Shaded  relief  does  not  aid  in  the 
solution  of  map  tasks  that  require  detailed  and  specific  terrain  infor¬ 
mation. 


vii 


Measurement  of  the  accuracy  and  speed  of  extraction  of  infor¬ 
mation  from  a  map  product  was  found  to  be  a  valuable  technique  for 
evaluation  of  experimental  map  products  and  a  valuable  alternative  to 
user  surveys  and  field  tests. 
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A  COMPARISON  OF  ALTERNATE  FORMATS  FOR  THE  PORTRAYAL  OF  TERRAIN  RELIEF  ON 
MILITARY  MAPS 


INTRODUCTION 


Maps  and  map  reading  are  essential  components  of  most  military 
operations,  and  their  role  has  become  increasingly  important  with  the 
emphasis  on  combined  arms  tactics  in  the  modern  U.S.  Army,  Actions  of 
infantry,  artillery,  aviation,  and  armor  must  be  closely  coordinated  and 
precise  positional  information  of  the  interacting  units  is  required.  To 
facilitate  accurate  positioning  it  is  necessary  to  have  maps  which 
present  terrain  information  in  a  form  that  is  clear  and  unequivocal  to 
the  user. 

In  an  effort  to  make  the  terrain  relief  information  on  maps  more 
perceptually  accessible  to  the  user,  the  U.S.  Army  Engineer  Topographic 
Laboratory  has  been  developing  new  map  formats  and  has  requested  ARI 
research  support  to  assess  the  effectiveness  of  these  experimental  map 
products.  This  report  presents  the  results  of  experiments  conducted  for 
this  evaluation  and  briefly  presents  the  research  methodology  which  ARI 
developed  for  that  purpose.  A  more  extensive  discussion  of  the  research 
methodology  developed  for  assessing  relief  map  legibility  is  presented 
by  Potash,  Farrell,  and  Jeffrey  (1978). 


Background 


Military  maps  must  present  information  about  terrain,  such  as  steep¬ 
ness,  shape,  and  elevation  of  land  forms,  on  a  two-dimensional  surface  in 
such  a  manner  that  it  can  be  perceived  quickly  and  accurately. 

For  approximately  the  last  50  years  the  means  of  depicting  terrain 
information  on  U.S.  Army  maps  has  been  contour  lines,  or  lines  drawn 
through  all  points  of  the  same  elevation  on  a  given  landform  with  the 
interval  between  adjacent  lines  reflecting  the  vertical  interval  on  the 
land.  Contour  lines  replaced  other  portrayal  techniques  because  they 
provide  the  most  precise  information  about  terrain.  Unfortunately, 
there  are  a  number  of  difficulties  with  contour  lines.  Some  of  these 
difficulties  are  cartographic:  cliffs  do  not  show  up  well;  small  terrain 
features  are  not  represented;  and  for  rough  terrain  with  extremes  of 
relief,  such  as  that  found  in  many  parts  of  the  western  U.S. ,  there 
simply  may  not  be  enough  space  on  the  map  sheet  for  the  number  of  contour 
lines  required  (Marsden,  1955).  A  more  serious  problem  with  contour 
lines,  however,  is  behavioral.  Many  map  users  have  difficulty  visual¬ 
izing  terrain  when  contour  lines  are  the  only  guide,  and  it  has  been 
found  that  many  military  map  users  report  to  hand  application  of 
additional  material  to  enhance  their  perception  of  the  terrain  portrayed 
(Skop,  1958).  In  addition,  in  many  military  tasks,  such  as  helicopter 
navigiation  at  low  altitudes,  there  is  insufficient  time  to  interpret 
contour  lines  accurately. 
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There  are  a  number  of  alternatives  and  supplements  to  contour  lines 
as  a  means  of  portraying  terrain  relief.  These  include:  (1)  hachures, 
or  linear  "hash"  marks,  to  indicate  the  presence  of  slopes;  (2)  the 
Tanaka  method,  a  system  of  modified  '■ontours;  (3)  digital  terrain  infor¬ 
mation,  or  spot  heights  in  numerical  forms;  (4)  perspective  drawing  or 
drawings  to  give  the  impression  of  space  or  depth;  (5)  relief  shading  or 
the  shading  of  slopes;  and  (6)  layer  or  gradient  tints  which  color  code 
different  altitude  zones.  After  conferring  with  U.S.  Army  Engineer 
Topographic  Laboratory  personnel,  it  was  concluded  that  for  military  map 
problems  it  would  be  undesirable  to  sacrifice  the  precision  of  contour 
lines,  so  supplements  rather  than  alternative  formats  were  considered. 
The  particular  contour  supplements  to  be  evaluated  were  selected  on  the 
basis  of  both  previous  research  and  the  availability  of  cartographic 
resources  for  generating  materials.  For  example,  the  quality  of  hachure 
are  highly  subject  to  the  skill  of  the  cartographic  artist  applying 
them,  and  easily  degenerate  into  "hairy  caterpillars"  (Robinson  and  Sale 
1969,  p.  175)  when  very  high  levels  of  expertise  are  not  employed. 
Similarly,  the  Tanaka  method  was  not  chosen  for  evaluation  because  the 
cartographic  techniques • required  are  not  available  on  a  widespread  basis 
As  a  result  of  these  considerations,  shaded  relief  and  layer  or  gradient 
tints  were  chosen  for  evaluation  as  supplements  to  contour  lines.  In 
Figure  1,  examples  of  unenhanced  contour  lines  and  of  contour  lines 
supplemented  with  shaded  relief  and  with  layer  tints  are  shown. 

In  shaded  relief,  light  is  presumed  to  come  from  one  direction,  and 
slopes  on  the  incident  side  of  a  hill  or  given  landform  are  unshaded, 
while  slopes  on  the  opposite  side  are  in  the  shade.  The  degree  of 
shading  varies,  with  the  steeper  slopes  being  darker.  In  gradient  or 
layer  tints,  terrain  is  color  coded  to  indicate  bands  of  elevation. 
Several  gradations  of  a  single  hue  may  be  used  or  these  may  be  different 
colors;  for  example,  500  to  1500  feet  may  be  represented  by  yellow,  1500 
to  2500  feet  by  tan,  and  all  elevation  above  2500  feet  by  brown.  In 
the  current  research  three  to  four  different  colors  were  used,  depending 
on  the  range  of  elevation  present  in  the  specific  map  segments. 

In  related  research  on  the  value  of  layer  tints  Kempf  and  Poock 
(1969)  used  12  tints,  varying  in  both  intensity  and  color,  to  compare 
contour  lines  versus  contour  lines  plus  layer  tints.  Their  experiment 
showed  that  the  addition  of  layer  tints  improved  performance  on  deter¬ 
mining  the  altitude  of  given  points,  but  had  a  detrimental  effect  on 
determining  grid  coordinates.  DeLucia  (1972)  also  found  that  the 
addition  of  relief  shading  had  a  detrimental  effect  on  the  accessibility 
of  two-dimensional  terrain  information. 

In  another  experiment,  Phillips,  DeLucia  and  Skelton  (1975)  compared 
four  types  of  maps:  contour  line,  contour  line  supplemented  with  shaded 
relief,  layer  tints,  and  digital  or  spot  height  maps.  An  interesting 
innovation  in  this  study  was  the  use  of  two  areas  of  the  Atlantic  sea 


Figure  1.  Examples  of  Three  Map  Formats;  Contour  Lines,  Shaded  Relief, 
and  Layer  Tints 


floor  to  produce  the  maps  and  thereby  control  for  previous  experience 
with  the  map  or  actual  terrain  as  a  contaminating  variable.  In  their 
results,  layer  tints  proved  best  for  judging  relative  height  and 
visualizing  the  landscape.  Contour  lines  and  contour  lines  supplemented 
with  relief  shading  proved  to  be  roughly  equal  for  all  tasks.  As 
expected,  digital  or  spot  height  maps  were  relatively  good  for  height 
judgment,  but  not  for  other  tasks.  Philips  et  al.  used  layer  tints  without 
contour  lines,  while  in  the  current  research,  contour  lines  were  supple¬ 
mented  by  layer  tints.  There  were  also  a  number  of  procedural 
differences . 

The  results  of  previous  research  suggested  that  both  the  maps  supple¬ 
mented  with  shaded  relief  and  those  supplemented  with  layer  tints  would 
be  superior  to  unenhanced  contour  lines.  It  was  also  predicted  that  the 
experimental  maps  would  be  less  advantageous  for  the  more  experienced 
map  users  who  were  more  familiar  with  the  standard  contour  line  format 
than  for  the  less  experienced  map  users.  The  present  research  sought  to 
test  both  of  these  hypotheses. 

The  Evaluation  of  Military  Maps 

The  primary  goal  of  the  research  was  to  evaluate  the  effectiveness 
of  the  alternate  techniques  for  the  portrayal  of  terrain  relief.  A 
secondary  goal  of  the  research  effort,  however,  was  to  develop  a  research 
methodology  for  the  evaluation  of  military  maps.  Until  recently,  alter¬ 
nate  map  formats  were  evaluated  by  either  surveys  of  user  preference  or 
by  field  tests.  Both  of  these  techniques  have  inherent  problems.  While 
surveys  are  relatively  inexpensive  and  also  provide  information  on  the 
degree  of  user  acceptance,  user  preference  surveys  have  questionable 
predictive  validity  for  performance  with  maps  unless  the  individuals 
questioned  have  actually  used  the  map  products  under  consideration 
(Wheaton,  Zavala,  and  Van  Cott,  1967).  One  study  found  that  the  prefer¬ 
ence  opinions  of  potential  users  were  useless  as  predictors  of  perfor¬ 
mance  under  daylight  conditions,  and  only  in  a  night  experiment  were 
subjective  judgments  of  preference  of  any  value  in  predicting  user 
performance  (Hill,  1974).  The  positive  results  at  night  were  most  likely 
due  to  the  effect  of  image  quality  on  perception  at  low  levels  of  illumi¬ 
nation.  Under  daylight  conditions,  however,  judgment  was  affected  by 
biases  due  to  the  esthetic  qualities  of  the  map,  personal  preferences 
due  to  previous  experience,  and  other  contaminating  factors. 

Field  tests  present  other  problems.  Objective  field  testing  is 
expensive  in  terms  of  time  and  dollars,  and  many  extraneous  variables 
which  are  difficult  to  control  are  present.  In  one  study  no  significant 
differences  were  found  between  map  types  because  of  the  large  number  of 
uncontrolled  variables  present  in  the  field  (MAPPRO  II,  1973).  If  only 
one  terrain  is  used,  the  question  arises  about  whether  the  results  are 
terrain  specific  or  if  generalizations  can  be  made  to  other  terrains. 
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Seasonal  effects  exist  and  must  be  controlled;  for  example,  land  naviga¬ 
tion  is  generally  easier  in  the  late  fall  and  winter  when  vegetation 
does  not  obscure  the  shape  of  land  forms  or  landmarks  such  as  small 
streams.  A  more  serious  contaminating  variable  is  tracks  made  through 
grass  and  other  vegetation  by  the  early  players.  These  tracks  persist 
for  a  considerable  length  of  time  and  affect  the  behavior  of  subsequent 
players . 

In  view  of  these  problems  with  the  more  traditional  approaches  to  map 
format  evaluation,  an  approach  was  developed  focused  on  the  information 
extracted  from  the  map.  Somewhat  similar  approaches  were  used  by  Kempf 
and  Poock  (1969),  tdio  measured  time  and  accuracy  in  determining  eleva¬ 
tion  and  grid  coordinates,  and  by  Phillips,  DeLucia  and  Skelton  (1975), 
who  required  participants  to  extract  12  types  of  information  from  maps. 
Both  the  approach  of  Kempf  and  Poock  (1969)  and  that  of  Phillips  (1975) 
have  some  similarities  to  the  current  approach. 

Eight  types  of  relief  information  normally  extracted  from  maps  by 
Army  users  were  identified,  based  upon  consultations  with  personnel  at 
the  U.S.  Army  Engineer  Topographic  Laboratory  and  other  representative 
military  users,  and  developed  into  subtests.  Map  segments  representing 
four  types  of  terrain  ranging  from  relatively  flat  to  mountainous  were 
used.  Answers  were  scored  for  both  accuracy  and  time.  Illustrations 
of  each  of  the  eight  terrain  formation  items  are  described  below; 
however,  a  more  extensive  description  is  provided  in  Potash,  Farrell, 
and  Jeffrey  (1978). 


Eight  Types  of  Terrain  Information 


1.  Landform  Identification.  The  participants  are  presented  with  map 
segments  with  arrows  overprinted,  and  shown  sketches  of  various  landforms, 
such  as  hills,  valleys,  spurs,  saddles,  or  depressions.  The  partici¬ 
pants  are  then  required  to  specify  which  of  the  representative  landforms 
is  portrayed  on  the  map  at  the  tip  of  each  arrow  (Figure  2). 


2.  Ridge-Valley  Identification.  In  this  task  the  participants  are 
presented  with  map. segments  that  contain  overprinted  lines.  The  parti¬ 
cipants'  task  is  to  determine  if  a  line  is  overprinted  on  a  valley  or  a 
ridge,  or  if  it  does  not  fall  into  either  category  (Figure  3). 


3.  Slope  Identification.  Slopes  may  be  convex,  concave  or  uniform.  In 
this  task  the  participants  are  presented  with  map  segments  with  arrows 
overprinted.  The  participant  is  asked  which  type  of  slope  is  indicated 
and  if  the  arrow  runs  up  or  down  hill  (Figure  4). 
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Figure  2. 


Sample  Landform  Identification 
sketches.  Overprinted  letters 
with  appropriate  landform. 


problem.  Map  segment  and  landform 
on  map  segments  must  be  matched 
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Figure  4.  Sample  Slope  Identification  problem.  Slopes  indicated  by  over 
printed  lines  must  be  matched  with  an  illustrated  slope  type. 
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4.  Identification  of  High  and  Low  Areas.  In  this  task  the  participants 
are  presented  with  map  segments  divided  into  64  grid  squares ,  each  of  the 
grid  squares  representing  an  area  1000  x  1000  meters.  The  participants 
must  specify  the  4  highest  and  the  4  lowest  grid  squares  (Figure  5). 

5.  Spot  Elevation  Problems.  Map  segments  with  several  points  indicated 
by  letters  are  presented  to  the  participants,  who  are  required  to 
determine  the  elevation  of  each  point  (Figure  6). 

6.  Vertical  Profile  Identification.  In  this  task  participants  are  pre¬ 
sented  with  maps  with  overprinted  lines  which  indicate  slopes.  The  parti¬ 
cipant  must  match  the  slope  indicated  on  the  map  with  one  of  three 
drawings  of  slope  angles  (Figure  7). 

7.  Terrain  Visualization.  Participants  are  shown  a  terrain  sketch  and 
map  segments  with  points  indicated.  The  participants  must  visualize  each 
of  the  points  indicated  on  the  map  and  determine  which  best  matches  the 
sketch  (Figure  8). 

8.  Defilade  or  Intervisibility.  Map  segments  with  points  indicated  by 
letters  are  presented  to  the  participants,  and  the  participants  are  asked 
to  determine  if  observers  located  at  selected  pairs  of  points  are  visible 
to  each  other  (Figure  9). 

Since  the  eight  subtests  directly  measure  the  accuracy  and  speed 
with  which  terrain  information  is  extracted  from  maps,  these  subtests 
are  inherently  valid.  The  basic  criteria  in  performance  with  a  map  is 
the  extraction  of  terrain  information.  Tests  which  contain  a  represen¬ 
tative  sample  of  the  behavior  domain  to  be  tested  have  content  validity 
(Anastasi,  1976,  p.  96-102  and  133-134).  The  eight  subtests  represent 
types  of  information  that  Army  personnel  must  extract  from  maps.  Thus 
these  eight  subtests  are  criterion  referenced  and  possess  content 
validity. 

Unfortunately,  content  validity  may  not  insure  predictive  validity. 

As  stated  earlier,  performance  in  the  field  with  a  map  is  a  complex 
multifaceted  task,  and  predictive  validity  cannot  be  assumed.  Therefore, 
as  part  of  the  research  program,  performanc  on  the  eight  subtests  was 
compared  with  field  performance  on  map  use  tasks. 


Research  Approach 


In  Experiment  I  a  total  of  48  Army  officers  and  NCOs  were  tested  for 
the  speed  and  accuracy  with  which  they  solved  terrain  information 
problems  in  each  of  the  eight  subtests.  While  there  was  a  considerable 
range  in  the  level  of  previous  map  experience  of  the  participants,  the 
overall  map-use  experience  of  the  group  was  somewhat  higher  than  that  of 
the  general  U.S.  Army  population. 
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contour  Interval  40  ft. 


Figure  5 


Sample  High-Low  Area  problem.  The  four  highest 
grids  on  overprinted  map  must  be  identified. 


Figure  7 


Sample  Vertical  Profile  problem.  Profile  indicated  by  overprinted 
line  on  map  sement  must  be  matched  with  a  line  drawing  illustrating 
vertical  profiles. 
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Experiment  II  was  essentially  a  replication  of  Experiment  I,  but  the 
overall  map  user  experience  level  was  lower.  The  participants  in  Experi 
ment  II  generally  had  been  in  the  Army  less  time  and  had  less  map  experi 
ence  than  the  participants  in  Experiment  I.  The  participants  in 
Experiment  II  were  chosen  so  that  results  of  the  research  effort  could 
be  generalized  to  a  much  larger  portion  of  the  Army  population.  The 
characteristics  of  the  participants  and  the  research  procedure  are 
presented  in  greater  detail  in  the  Method  section. 

A  field  validation  was  also  conducted  to  insure  that  results  of  the 
measures  used  in  Experiments  I  and  II  would  generalize  to  performance  in 
the  field.  In  this  experiment  only  the  standard  contour  line  map  was 
evaluated  and  the  performance  of  participants  on  the  assessment  test  was 
compared  with  performance  on  field  criteria. 

Research  Objectives 

(1)  To  evaluate  two  experimental  map  formats,  using  shaded  relief 
and  layer  tints  as  supplements  to  contour  lines,  and  compare  them  to 
standard  U.S.  Army  maps. 

(2)  To  assess  the  effects  of  user  experience  with  each  type  of  map, 
and  determine  if  alternate  map  formats  are  preferable  with  less  experi¬ 
enced  personnel. 

(3)  To  determine  the  effectiveness  of  a  method  of  evaluating  map 
products  by  measuring  the  speed  and  accuracy  with  which  information  is 
extracted  from  the  map. 

METHOD 


Experiment _ ^ 

Participants.  A  total  of  48  male  Army  officers  and  NCOs  ranging  in 
rank  from  sergeant  through  major  participated  in  this  experiment. 
Although  the  ..evel  of  experience  in  use  of  the  standard  Army  contour 
line  map  varied,  the  overall  level  of  experience  was  somewhat  higher 
than  the  general  Army  population.  Participants  were  obtained  from  the 
Second  Armored  Division.  This  particular  unit  uses  maps  for  land  naviga 
tion  more  often  than  many  other  units.  Approximately  two-thirds  of  the 
participants  were  Ranger  qualified  and  had  received  advanced  instruction 
in  map  use  and  land  navigation.  Those  participants  who  were  Ranger 
qualified  and  had  used  maps  in  exercises  regularly  within  the  last  two 
years  were  considered  highly  experienced;  those  participants  who  had 
Ranger  training  but  had  not  used  maps  in  exercises  within  two  years  were 
considerd  moderate  in  experience;  and  those  who  had  not  received  Ranger 


training  and  had  not  used  maps  in  field  exercises  in  two  years  were 
considered  low  in  experience.  Fourteen  participants  were  in  the  low 
category,  19  in  the  moderate,  and  15  in  the  highly  experienced  category. 

Performance  Measures.  A  test  containing  eight  subtests  was  created, 
corresponding  to  the  eight  critical  types  of  relief  information  discussed 
earlier,  to  measure  the  extraction  of  information  from  each  map  format. 

The  eight  types  are:  landform  identification,  ridge-valley  identification, 
identification  of  slopes,  identification  of  high  and  low  ground,  determi¬ 
nation  of  elevation  of  spots,  vertical  profile  identification,  terrain 
visualization,  and  intervisibility  or  defilade.  Each  subtest  was  scored 
on  both  speed  and  accuracy.  Map  segments  representing  4000  meter  by  4000 
meter  grid  squares  accompany  questions,  except  for  the  "identification  of 
high-low  area"  problem  which  uses  8000-meter  grid  squares.  The  map 
segments  are  taken  from  the  standard  scale  or  1:50,000  map,  and  four 
terrain  contour  intervals  were  used:  10,  20,  40,  and  80  feet. 

Procedure .  Participants  were  tested  in  small  groups  of  either  six  or 
eight  in  a  large  room  with  widely  spaced  tables  and  one  participant  per 
table.  Participants  were  randomly  assigned  to  seats,  and  the  format 
placed  at  a  given  seat  was  varied  on  a  daily  basis.  Participants  were 
instructed  to  work  as  quickly  and  as  accurately  as  possible.  They  were 
instructed  to  take  a  rest  break  if  they  became  fatigued,  but  to  take  the 
break  only  at  the  end  of  a  problem.  Stopwatches  were  given  to  each 
participant  to  determine  the  amount  of  time  required  to  solve  each  problem 
and  they  were  instructed  to  indicate  the  time  required  to  solve  each 
problem  with  the  answers  on  their  answer  sheet.  Each  participant  worked 
with  only  one  problem  at  a  time. 

Experiment  II 

Participants ■  A  total  of  30  participants  were  used  in  Experiment  II. 
The  participants  were  obtained  from  initial  entry  officer  candidate 
students.  Overall  experience  level  was  somewhat  lower  than  that  of  the 
general  Armv  population,  although  five  participants  did  have  extensive 
prior  experience. 

Procedure .  Ten  different  participants  were  tested  each  day  in  a 
large  room  for  three  consecutive  days.  The  map  formats  placed  at  the 
seats  were  altered  each  day  and  seats  were  assigned  to  obtain  the 
required  number  of  participants  in  each  particular  experimental  condition. 


RESULTS 


Overview 


In  this  section,  results  for  the  major  variables,  map  format  and  user 
experience,  are  summarized  in  terms  of  both  accuracy  and  time  scores  for 
both  Experiments  I  and  II  (Tables  1  through  4).  Appendixes  A-D  contain 
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the  results  in  greater  detail,  with  mean  accuracy  and  median  time  scores 
for  each  experience  level  as  well  as  analysis  of  variance  tables. 

Results  from  the  field  validation  experiment  to  assess  the  predictive 
validity  of  the  subtests  are  presented  in  Appendix  E. 

Experiment  I 

Since  there  was  no  overall  performance  score,  separate  two-way 
analyses  of  variance  (Map  Format  x  Experience  Level)  were  conducted  for 
each  of  the  eight  subtests  for  both  accuracy  and  time  scores.  Therefore, 
there  are  16  two-way  analyses  of  variance.  Of  the  two  major  variables, 
map  format  and  user  experience,  map  format  is  the  more  important. 

Results  on  the  map  format  variable  are  summarized  for  accuracy  scores 
in  Table  1  and  for  time  scores  in  Table  2.  In  Table  1,  statistically 
significant  effects  for  map  format  can  be  seen  on  the  landform  identifica¬ 
tion,  the  ridge-valley  identification  and  the  spot  elevation  subtests. 

On  these  three  subtests,  the  map  format  using  layer  tints  was  superior 
on  two:  ridge-valley  identification  and  spot  elevation.  Inspection  of 
Table  1  shows  that  the  accuracy  scores  on  the  contour  line  and  layer 
tint  formats  are  relatively  close  to  each  other,  and  that  on  all  eight 
subtests,  the  map  format  using  shaded  relief  was  poorest. 

Table  2  is  a  summary  of  the  results  using  time  required  to  solve  the 
problem  as  the  scoring  criterion.  Inspection  of  Table  2  indicates  that  with 
layer  tint  format  the  shortest  time  was  required  to  solve  the  problem  for 
seven  of  the  eight  subtests,  the  only  exception  being  terrain  visualiza¬ 
tion.  On  only  two  of  the  subtests,  however,  was  the  effect  statistically 
significant:  ridge-valley  identification,  and  identification  of  high  and 
low  areas . 

The  other  major  variable,  user  experience  level,  concerned  two  signifi¬ 
cant  issues,  (1)  main  effect  due  to  experience  and  (2)  interaction  effects 
between  map  format  and  experience  levels.  The  results  regarding  experi¬ 
ence  are  presented  in  Table  3. 

Experience  level  of  the  user  produced  a  significant  main  effect  on 
all  eight  subtests  when  accuracy  scores  were  used  as  a  criterion  for  per¬ 
formance.  Using  time  scores  as  a  criterion,  however,  produced  only 
three  significant  effects.  There  were  no  statistically  significant  inter¬ 
action  effects  between  experience  level  of  user  and  map  format  in  any  of 
the  16  two-way  analyses  of  variance  in  Experiment  I. 

Experiment  II 


In  the  data  on  Experiment  II,  within-cell  variance  was  higher  than 
in  Experiment  I  and  fewer  statistically  significant  effects  were  found. 
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TABLE  1 

Experiment  I:  Summary  of  Significant  Effects  on  Map  Format 
for  Mean  Accuracy  Scores 


1 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

I 

1. 

Land form  Ident 

13.64 

12.4 

13.12* 

Ek  -  *  1 

2. 

Ridge-Valley  Ident 

23.86 

20.67 

25. 19** 

3. 

Slope  Ident 

12. 14 

10.28 

11.69 

r  '  1 

4. 

Ident  of  High-Low 

Areas  47.6 

46.8 

48.44 

b 

5. 

Spot  Elevation 

23.14 

16.56 

22.56** 

6. 

Vertical  Profile 

8.5 

6.78 

7.69 

7. 

Visualization 

9.5 

8.34 

9. 13 

8. 

Defilade 

20.86 

19.44 

19.69 

\ 

* 

p<-05 

■  i 

;  1 

‘  l 

★* 

p<.01 

TABLE  2 

Experiment 

I:  Summary  of  Significant  Effects  on  Map  Format 

for  Median  Time  Scores 

' 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Land  form  Ident 

1.02 

1.04 

.76 

2. 

Ridge-Valley  Ident 

2.36 

2.  37 

1.71* 

3. 

Slope  Ident 

1.34 

1.36 

1.19 

l 

4. 

Ident  of  High-Low 

Areas  4.23 

3. 13 

2. 51** 

» 

5. 

Spot  Elevation 

4.51 

4. 14 

3.  34 

6. 

Vertical  Profile 

1.87 

1.7 

1.57 

I  ^ 

u 

7. 

Visualizat ion 

.88 

.78 

1.0 

1 

8. 

Defilade 

2.68 

2.46 

1.9 

*  p<. 05 

**p<. 01 
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Table  3 

Experiment  I:  Statistically  Significant  Effects  From 
User  Experience  Levels 


Accuracy 

Scores 

Time 

Scores 

Subtest 

Main 

Effect 

Interaction 
With  Format 

Main 

Ef  feet 

Interaction 
With  Format 

1. 

Land  form  Ident 

.01 

NS 

NS 

NS 

2. 

Ridge-Valley  Ident 

.01 

NS 

NS 

NS 

3. 

Slope  Ident 

.05 

NS 

.05 

NS 

4. 

Ident  of  High-Low  Areas 

.01 

NS 

NS 

NS 

5. 

Spot  Elevation 

.01 

NS 

.05 

NS 

6. 

Vertical  Profile 

.01 

NS 

.05 

NS 

7. 

Visualizat ion 

.05 

NS 

NS 

NS 

8. 

Def i lade 

.01 

NS 

NS 

NS 

19 


In  the  accuracy  scores  of  Experiment  II,  no  statistically  significant 
effects  were  found  from  either  the  map  format  or  the  user  experience 
level  variables.  Trends  were  similar  to  Experiment  I  in  that  the  shaded 
relief  format  produced  the  lowest  mean  score  on  seven  of  the  eight  sub¬ 
tests  with  the  low  experience  groups,  and  the  lowest  mean  scores  on  all 
eight  subtests  in  the  higher  experience  group.  Although  these  effects 
were  not  statistically  significant,  such  consistent  trends  should  not  be 
ignored.  These  trends  may  be  seen  by  inspecting  tables  in  Appendix  C. 

Statistically  significant  effects  were  found  in  the  time  scores  for 
both  the  map  format  and  user  experience  level  variables.  In  map  format, 
the  landform  identification  and  the  terrain  visualization  subtests  pro¬ 
duced  significant  effects.  A  summary  of  the  effects  obtained  with  time 
scores  in  Experiment  II  is  presented  in  Table  4. 

For  the  low  experience  group,  shaded  relief  was  the  poorest  format, 
except  on  the  terrain  vi sual i zat ion  subtest.  Shaded  relief  was  somewhat 
better  for  the  higher  experience  group,  as  can  be  seen  in  greater  detail 
in  Appendix  D,  Tables  D-l  and  D-2. 

The  experience  variable  produced  significant  effects  on  the  landform 
identification  and  the  ridge-valley  identification  subtests  in  time 
scores.  In  the  landform  identification  time  score  results,  a  significant 
interaction  occurred  between  map  format  and  experience  (p<.01).  In  the 
total  of  31  tests  for  interactions  between  map  format  and  experience  in 
Experiments  I  and  II  this  was  the  only  significant  interaction. 

DISCUSSION 

Overview 

In  providing  an  interpretation  of  the  data  acquired  in  Experiments  I 
and  II,  three  major  issues  must  be  considered:  (1)  effect  of  map  format; 
(2)  effect  of  user  experience;  and  (3)  interaction  effects  between  map 
format  and  experience.  Several  aspects  of  each  of  these  variables  must 
be  considered.  If  a  particular  map  format  is  superior  for  some  subtests 
but  not  for  others,  this  will  affect  the  generalizations  that  can  be  made 
about  that  particular  format.  Effects  of  experience,  and  interactions 
between  map  format  and  experience,  must  also  be  discussed.  Interactions 
are  particularly  important  because  a  large  number  of  significant  inter¬ 
actions  would  indicate  that  a  particular  map  format  was  more  effective 
at  some  experience  levels  than  others,  and  this  would  also  affect  the 
generalizations  that  could  be  made  from  the  data.  Two  remaining  issues 
must  be  addressed  in  discussing  the  experimental  results:  (1)  implica¬ 
tions  of  the  research  for  map  design  and  (2)  implications  for  evaluating 
cartographic  products.  This  last  issue  is  discussed  in  Appendix  E,  where 
the  predictive  validity  of  the  eight  subtests  is  addressed. 
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TABLE  4 


Experiment  II:  Summary  of  Significant  Effects  on  Map  Format 

for  Time  Scores 


Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Land  form  Ident 

1.24 

1.13 

.95** 

2. 

Ridge-Valley  Ident 

1.74 

2.  18 

1.  76 

3. 

Slope  Ident 

1.01 

1.46 

1.  13 

4. 

Ident  of  High-Low  Areas 

2.48 

2.83 

2.  31 

5. 

Spot  Elevation3 

Insuf  f i c  i  ent 

data  on  This  Subtest 

6. 

Vertical  Profile 

2.42 

1.  51 

2.45 

7. 

Visualizat ion 

.73 

.61 

1.0** 

8. 

Def i lade 

2.26 

1.93 

2.02 

aThe  spot  elevation  of  32  points  must  be  determined  on  this  subtest,  and 
due  to  the  generally  low  experience  level  of  the  participants  in  this 
experiment,  many  of  them  were  unable  to  complete  enough  items  to  obtain 
meaningful  time  scores  on  this  subtest. 

**p< . 0  1 
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Effects  of  Map  Format  on  Accuracy  in  Experiments  I  and  II 


In  Experiments  I  and  II  the  most  important  issue  is  the  effect  of  map 
format — whether  or  not  the  supplements  of  shaded  relief  or  layer  tints  to 
standard  U.S.  Army  contour  line  maps  result  in  superior  performance.  In 
Experiment  I,  using  accuracy  scores  as  criteria,  statistically  significant 
effects  were  found  on  three  subtests:  landform  identification,  ridge-valley 
identification,  and  spot  evaluation.  On  each  of  these  subtests  the  differ¬ 
ence  was  small  between  the  map  format  using  layer  tints  as  a  supplement  and 
the  standard  contour  line  map.  Both  of  these  formats,  however,  were 
superior  to  the  format  using  shaded  relief  as  a  supplement.  A  general 
trend  in  this  direction  is  also  evident  on  the  remaining  five  subtests; 
however,  the  effects  did  not  reach  statistically  significant  levels. 

These  general  effects  occurred  for  all  experience  levels  in  Experiment  I. 

Experiment  II  was  rather  similar  to  Experiment  I  but  was  conducted  on 
a  population  with  a  generally  lower  experience  level.  It  was  thought  that 
the  relatively  poor  performance  found  with  the  shaded  relief  supplement 
might  have  been  due  to  the  fact  that  participants  in  Experiment  I  were 
somewhat  more  experienced  in  using  maps  than  were  the  general  U.S.  Army 
population.  This  greater  experience  with  the  standard  U.S.  Army  contour 
line  map  might  have  created  a  bias  in  favor  of  the  standard  map,  so 
participants  with  an  experience  level  somewhat  lower  than  that  of  the 
general  U.S.  Army  population  were  used  in  Experiment  II.  However,  using 
accuracy  scores  as  criteria,  no  statistically  significant  effects  were 
found  on  map  format.  Mean  scores  on  the  layer  tint  format  were  consis¬ 
tently  higher  (although  not  statistically  significant)  than  on  shaded 
relief  format  on  seven  of  the  eight  subtests  in  Experiment  II.  Such  a 
consistent  trend  should  not  be  ignored.  These  results  with  accuracy 
scores  in  Experiment  II  reflect  the  same  trend  found  in  Experiment  I. 

For  most  Army  map  use  problems  involving  specific  terrain  relief  infor¬ 
mation,  layer  tints  appear  superior  to  shaded  relief  as  a  supplement  to 
contour  lines.  This  replication  of  the  effect  found  in  Experiment  I  with 
a  different  overall  experience  level  among  the  participants  suggests 
that  the  effect  is  a  general  one. 

Effects  of  Map  Format  on  Time  Scores  in  Experiments  I  and  II 

In  the  time  data  for  Experiments  I  and  II,  results  were  generally 
similar  to  the  accuracy  data,  but  the  map  format  using  layer  tints  to 
supplement  contour  lines  was  superior  to  both  shaded  relief  or  unenhanced 
contour  lines.  In  Experiment  I,  significant  effects  occurred  only  in 
ridge-valley  identification  and  identification  of  high  and  low  areas.  In 
both  of  these  subtests,  layer  tints  resulted  in  faster  problem  solving 
time.  In  the  subtests  of  Experiment  I,  layer  tints  required  the  shortest 
solution  time  on  at  least  six  of  the  eight  subtests  regardless  of  the 
experience  group.  In  Experiment  II,  the  results  using  time  data  were 
somewhat  less  clear.  For  the  low  experience  groups,  layer  tints  and 
standard  contour  lines  were  better  than  shaded  relief  on  landform  identifi¬ 
cation,  but  for  the  higher  experience  group  the  standard  contour  line  map 
was  poorest. 


A  statist ica lllv  significant  effect  in  the  time  scores  favored  shaded 
relief  on  the  visualization  task.  This  effect  cannot  be  considered  a 
sampling  error  or  a  fluke.  Examination  of  the  visualization  subtests 
results  in  Experiment  I  (Tables  1  and  2)  indicates  a  slight  effect  favoring 
shaded  relief  when  both  time  and  accuracy  are  used  as  criteria.  Once 
again,  a  trend  is  evident.  The  visualization  subtest  differs  from  the 
others  in  that  the  participant  is  required  to  visualize  an  area  rather 
than  answer  questions  concerning  precise  detail  as  in  the  other  seven 
subtests  . 

Effects  of  User  Experience  Level 


User  experience  level  had  a  more  powerful  effect  on  accuracy  than  on 
time  in  Experiment  I.  This  effect  was  not  replicated  in  Experiment  II 
because  of  the  high-within  cell  variance  on  accuracy  scores  from  the 
relatively  less  experienced  participants.  The  effect  in  Experiment  I, 
however,  demonstrated  that  greater  levels  f  experience  resulted  in  more 
accurate,  but  not  necessarily  faster,  solutions  of  terrain  relief  problems. 
An  important  implication  of  this  finding  is  that  speed  of  solving  map 
tasks  may  be  less  sensitive  than  accuracy  as  a  measure  of  human  perfor¬ 
mance  with  maps . 

Interactions  Between  Map  Format  and  User  Experience  Level 

In  the  combined  results  of  Experiments  I  and  II,  there  were  31 
statistical  tests  for  interaction  between  the  two  main  effects,  map  format 
and  user  experience  level.  Only  one  statistically  significant  inter¬ 
action  occurred  in  the  31  tests.  The  lack  of  strong  interaction  effects 
suggests  that  while  the  less  experienced  participants  have  lower  perfor¬ 
mance  levels  than  the  more  experienced,  the  same  relative  patterns  occur 
for  different  experience  levels.  This  result  suggests  that  special 
purpose  maps  are  not  necessary  for  terrain  problems  with  less  experienced 
participants . 

Implications  for  the  Design  of  Maps 

These  results  have  several  implications  for  the  design  and  evalua¬ 
tion  of  military  and  commercial  maps.  In  general,  the  use  of  layer  tints 
to  enhance  contour  lines  made  relief  information  the  most  perceptually 
accessible  to  users.  This  was  true  for  most  of  the  measures  which 
involved  extracting  highly  detailed  information  typical  of  Army  map  tasks. 
The  use  of  shaded  relief  increased  the  amount  of  time  required  to  solve 
problems  with  no  increase  in  accuracy,  except  on  the  terrain  visualiza¬ 
tion  measure.  This  suggests  that  relief  shading  has  some  value  in  giving 
a  fast  general  impression  of  the  terrain  characterist ics  in  a  given  area, 
but  not  for  questions  involving  more  precise  terrain  information. 
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One  important  result  is  that  when  contour  lines  were  supplemented 
with  shaded  relief,  which  was  intended  to  enhance  the  perception  of 
terrain  relief,  there  was  an  increase  in  the  time  required  to  solve  the 
problems  with  no  corresponding  increase  in  accuracy  for  most  of  the  sub¬ 
tests.  This  unexpected  finding,  consistent  with  the  work  of  other 
investigators  such  as  Hill  (1974),  suggests  that  new  cartographic 
products  require  objective  evaluation.  It  is  likely  that  shaded  relief 
impeded  performance  because  of  a  cluttering  effect,  and  since  such 
effects  often  cannot  be  foreseen,  empirical  testing  of  new  military  map 
products  is  advisable. 


SUMMARY 

The  primary  goal  of  this  research  effort  was  to  evaluate  two  experi¬ 
mental  map  formats  designed  to  enhance  the  perceptual  accessibility  of 
terrain  relief  information.  The  two  experimental  map  formats  supple¬ 
mented  contour  lines  with  either  shaded  relief  or  layer  tints.  The 
first  experiment  involved  participants  who  had  relatively  more  experience 
with  maps  than  most  U.S.  Army  personnel.  To  insure  that  the  results  of 
research  would  generalize  to  the  entire  U.S.  Army  population,  a  group 
with  less  experience  was  used  in  a  second  experiment . 

For  most  of  the  eight  tvpes  of  terrain  information  that  subjects  were 
required  to  extract  from  maps,  the  map  format  using  layer  tints  to  supple¬ 
ment  contour  lines  resulted  in  faster  problem  solving  with  no  loss  in 
accuracy.  The  use  of  shaded  relief  did  not  increase  the  map  reading  speed 
and  sometimes  resulted  in  a  decrease  in  accuracy  of  extracting  the 
specific  terrain  information  typical  of  most  U.S.  Army  map  use  problems. 
Shaded  relief  was  of  some  value,  however,  for  giving  a  general  impression 
of  the  topography  of  a  given  area.  More  experienced  personnel  solved  map 
problems  more  accurately  but  not  necessarily  faster  than  inexperienced 
personnel.  In  31  tests  for  interactions  between  map  format  and  experi¬ 
ence,  only  one  significant  interaction  occurred.  A  map  format  that  is 
effective  at  one  experience  level  is  also  effective  at  another,  and 
special  purpose  maps  for  map  users  with  low  experience  do  not  appear 
necessary. 

A  secondary  goal  of  the  research  effort  was  to  further  develop  the 
research  methodology  for  evaluating  map  formats.  Traditional  techniques 
for  map  evaluation — user  surveys  and  field  tests — have  limitations. 
Therefore,  a  technique  was  developed  which  measures  the  speed  and 
accuracy  with  which  information  is  extracted  from  a  map  product.  This 
technique  was  found  to  be  a  useful  method  for  evaluating  the  effective¬ 
ness  of  experimental  maps. 
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APPENDIX  A 


/ 


ACCURACY  EFFECTS  IN  EXPERIMENT  I 
Table  A-l 


Mean  Accuracy  Scores  in  Experiment  I,  Low  Experience  Group 


Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Landform  Ident 

11.75 

10.25 

12.5 

2. 

Ridge-Valley  Ident 

18.25 

13.50 

21.8 

3. 

Slope  Ident 

10.25 

7.5 

11.0 

4. 

Ident  of  High-Low  Areas 

44.0 

40.5 

47.5** 

5. 

Spot  Elevation 

20.75 

8.  75 

19.  3 

6. 

Vertical  Profile 

6.5 

4.75 

5.83 

7. 

Visualization 

6.0 

7.0 

8.3* 

8. 

Defilade 

20.0 

16.25 

18.17** 

★ 

p<.05  level 

pC.Ol 

TABLE  A- 2 

Mean  Accuracy  Scores 

in  Experiment  I, 

Moderate  Experience 

Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Landform  Ident 

14.3 

12.0 

13.  14 

2. 

Ridge-Valley  Ident 

26.17 

21.83 

26.0 

3. 

Slope  Ident 

13.0 

10.  3 

11.29** 

4. 

Ident  of  High-Low  Areas 

48.3 

46.  5 

47.7 

5. 

Spot  Elevation 

25.67 

19.17 

23.57 

6. 

Vertical  Profile 

10.0 

7.  17 

8.0 

7. 

Visualization 

10. 10 

8.67 

9.43* 

8. 

Defilade 

20.3 

20.17 

20.0** 

*  p< .05 

**p<.01 
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TABLE  A- 3 


Mean  Accuracy  Scores 

In  Experiment  I, 

High  Experience 

Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Landform  Ident 

14.5 

13.88 

14.33 

2. 

Ridge-Valley  Ident 

26.0 

23.39 

30.67 

3. 

Slope  Ident 

12.75 

11.87 

14.0* 

4. 

Ident  of  High-Low  Areas 

50.0 

50.3 

52.0 

5. 

Spot  Elevation 

21.75 

18.5 

26.67 

6. 

Vertical  Profile 

8.25 

7.5 

10.67 

7. 

Visualization 

12.25 

8.88 

10.0* 

8. 

Def i lade 

22.5 

20.5 

22.0** 

★ 

p< .  05 

** 

p<.01 

TABLE  A-4 


Analys i s 

of  Variance 
Land  form 

for  Accuracy, 
Identi ficat ion 

Experiment  I 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

9.47 

3.31 

.05 

Expert  ence 

2 

27.71 

9.67 

.01 

A  x  B 

4 

2.02 

.70 

NS 

Within  cells 

39 

2.86 

Total  47 
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TABLE  A- 5 


Analysis  of  Variance  for  Accuracy,  Experiment  I 
Ridge  -  Valley  Identification 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

135.51 

6.93 

.01 

Experience 

2 

320.03 

14.26 

.01 

A  X  B 

4 

9.40 

.42 

NS 

Within  cells 

39 

22.45 

Total 

47 

TABLE  A- 6 

Ana lys i s 

of  Variance  for  Accuracy, 
Slope  Identification 

Experiment  I 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

26.01 

2.  78 

NS 

Experience 

2 

44. 36 

4.  74 

.05 

A  X  B 

4 

5.  20 

.  56 

NS 

Within  cells 

39 

9.  36 

Tot  a  1 

47 
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TABLE  A- 7 


Analysis  of  Variance  for  Accuracy,  Experiment  1 
Identi f icat ion  of  High  and  Low  Areas 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

40.23 

1.  35 

.NS 

Experience 

2 

166.23 

5.60 

.01 

A  X  B 

4 

14.66 

.49 

NS 

Within  cells 

39 

29.70 

Total 

47 

TABLE  A- 8 

Analysis 

of  Variance  for  Accuracy, 
Slope  Identification 

Experiment  1 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

273.12 

13.65 

.01 

Exper i ence 

2 

192.38 

9.62 

.01 

A  X  B 

4 

33.83 

1.  79 

NS 

Within  cells 

39 

20.0 

Total 

47 

'V, 
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TABLE  A-9 


Analysis  of  Variance  for  Accuracy,  Experiment  I 
Vertical  Profile 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

14.68 

1.95 

NS 

Experience 

2 

41.60 

5.54 

.01 

A  X  B 

4 

6.37 

.85 

NS 

Within  cells 

39 

7.51 

Total 

47 

TABLE  A- 10 

Analysis 

of  Variance  for  Accuracy, 
Terrain  Visualization 

Experiment  I 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

6.58 

.79 

NS 

Experience 

2 

40.73 

4.87 

.05 

A  X  B 

4 

8.31 

.99 

NS 

Within  cells 

39 

8.  36 

Total 

47 
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TABLE  A- 11 


Analysis  of  Variance  for  Accuracy,  Experiment  I 

Def i lade 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

14.23 

1.98 

NS 

Experience 

2 

45.72 

6.35 

.01 

A  X  B 

4 

4.13 

.57 

NS 

Within  cells 

39 

7.20 

Total 

47 

TABLE  A-12 

Summary  of  Significant  Accuracy 

Effects  in  Experiment  I 

Main 

Effects 

Interac  t ion 

Subtest 

Map  Format 

Experience 

1. 

Land  form  Ident 

NS 

.05 

NS 

2. 

Ridge-Valley  Ident 

NS 

.01 

NS 

3. 

Slope  Ident 

.01 

.01 

NS 

A. 

Ident  of  High-Low 

Areas 

NS 

.01 

NS 

5. 

Spot  Elevation 

NS 

.05 

NS 

6. 

Vertical  Prof i le 

NS 

NS 

NS 

7. 

Visual i zat i on 

.05 

.01 

NS 

8, 

Defi  lade 

.01 

.01 

NS 
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MEDIAN  TIME  EFFECTS  IN  EXPERIMENT  I 
TABLE  B-l 


Median  Time  Scores 

in  Experiment  I, 

Low  Experience  Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Land form  Ident 

1.42 

.86 

.92 

2. 

Ridge-Valley  Ident 

3.01 

2.16 

1.96* 

3. 

Slope  Ident 

1.72 

1.72 

1.29 

4. 

Ident  of  High-Low  Areas 

4.78 

2.67 

2.9** 

5. 

Spot  Elevation 

6.32 

4. 18 

4.  12 

6. 

Vertical  Profile 

1.25 

.77 

1.38 

7. 

Visualizat ion 

.68 

.86 

1.2 

8. 

Defilade 

2.57 

1.88 

1.75 

*p<  .05  level 

**p<  .01 

Median  Time  Scores 

TABLE  B-2 

in  Experiment  I, 

Moderate  Experience 

Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Land  form  Ident 

.84 

1.32 

.68 

2. 

Ridge-Valley  Ident 

2.23 

2.99 

1.58* 

3. 

Slope  Ident 

1.17 

1.64 

1.  17 

4. 

Ident  of  High-Low  Areas 

3.54 

4.  1 

2. 18** 

5. 

Spot  Elevation 

3.59 

5.  1 

2.99 

6. 

Vertical  Profile 

2.13 

2.0 

1.4 

7. 

Visualization 

.89 

.81 

.89 

8. 

Def i lade 

2.49 

3.08 

1.86 

*p<  .05  level 

**p<  .01 
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TABLE  B-3 


Median  Time  Scores 

in  Experiment  I. 

High  Experience  Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tints 

1. 

Landform  Ident 

.89 

.92 

.61 

2. 

Ridge-Valley  Ident 

1.9 

2.0 

1.51* 

3. 

Slope  Ident 

1.24 

.98 

1.03 

4. 

Ident  of  High-Low  Areas 

4.72 

2.63 

2.48** 

5. 

Spot  Elevation 

4.09 

3.39 

2.89 

6. 

Vertical  Profile 

2. 11 

1.95 

2.35 

7. 

Visualization 

1.08 

.71 

.87 

8. 

Defilade 

3.14 

2.28 

2.29 

*p<  .05  level 

**p<  .01 


TABLE  B-4 

Analysis  of  Variance  for  Median  Times,  Experiment  I, 
Landform  Identification 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

.44 

2.37 

NS 

Experience 

2 

.24 

1.29 

NS 

A  X  B 

4 

.35 

1.86 

NS 

Within  cells 

39 

.9 

Total 

47 
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TABLE  B-5 


Analysis  of  Variance  for  Median  Times,  Experiment  1, 
Ridge-Valley  Identification 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

2.32 

3.36 

.05 

Experience 

2 

1.28 

1.86 

NS 

A  X  B 

4 

.95 

1.38 

NS 

Within  cells 

39 

.69 

Total 

47 

Ana  lysis 

TABLE  B-6 

of  Variance  for  Median  Times,  Experiment  I 
Slope  Identification 

J 

Source 

df 

MS  F 

P< 

Map  Format 

2 

.31  1.29 

NS 

Experience 

2 

.89  3.66 

.05 

A  X  R 

4 

.22  .89 

NS 

Within  cells 

39 

.24 

Total 

47 
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TABLE  B-7 


1 


Analysis 

of  Variance 
High  and 

for  Median 
Low  Areas 

Times,  Experiment 

I, 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

12.61 

10.01 

.01 

Experience 

2 

.15 

.12 

NS 

A  X  B 

4 

3.13 

2.49 

NS 

Within  cells 

39 

1.26 

Total 

47 

Analysis 

TABLE  B-8 

of  Variance  for  Median  Times,  Experiment  I, 
Spot  Elevation 

Source 

df 

MS  F 

P< 

Map  Format 

2 

6.73  3.20 

NS 

Experience 

2 

7.68  3.64 

.05 

A  X  B 

4 

4.20  1.99 

NS 

Within  cells 

39 

2.11 

Total 

47 
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TABLE  B-9 


Analysis  of  Variance  for  Median  Times,  Experiment  I, 
Vertical  Profile 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

.24 

.26 

NS 

Experience 

2 

3.87 

4.12 

.05 

A  X  B 

4 

.60 

.63 

NS 

Within  cells 

39 

.94 

TABLE  B-10 

Analysis  of  Variance  for  Median  Times, 
Terrain  Visualization 

Experiment  I, 

Source 

df 

MS 

F  P< 

Map  Format 

2 

.  14 

.83  NS 

Experience 

2 

.01 

.06  NS 

A  X  B 

4 

.19 

1 

.15  NS 

Within  cells 


39 


.17 


TABLE  B-l 1 

Analysis  of  Variance  for  Median  Times,  Experiment  I, 
Defilade 


[  j 

Source 

df 

MS 

F 

P< 

b\ 

Map  Format 

2 

2.09 

1.91 

NS 

*  \ 

Experience 

2 

1.02 

.93 

NS 

*■-  I 

►  ;  \ 

A  X  B 

4 

.93 

.84 

NS 

i 

"  i 

Within  cells 

39 

1.10 

] 

f 

Total 

47 

TABLE  B-l 2 

Summary 

of  Significant  Time  Effects 

in  Experiment  I 

• 

i 

Main  Effects 

Interact  ion 

i 

Subtest 

Map  Format 

Experience 

Landform  Ident 
Ridge-Valley  Ident 
Slope  Ident 

Ident  of  High-Low  Areas 

5.  Spot  Elevation 

6.  Vertical  Profile 

7.  Visualization 

8.  Defilade 


APPENDIX  C 

ACCURACY  EFFECTS  IN  EXPERIMENT  II 
TABLE  C-l 


Mean  Accuracy 

Scores  in  Experiment 

II,  Low  Experience 

Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tint 

1. 

Landform  Ident 

12.44 

12.12 

13.25 

2. 

Ridge-Valley  Ident 

23.3 

20.5 

25.25 

3. 

Slope  Ident 

12.44 

11.0 

11.87 

4. 

Ident  of  High-Low  Areas 

41  .66 

46.0 

-P' 

00 

u> 

-■J 

5. 

Spot  Elevation 

21.2 

18.75 

20.75 

6. 

Vertical  Profile 

9.2 

8.43 

7.62 

7. 

V isual izat ion 

8.3 

6.67 

9.12 

8. 

Defilade 

19.3 

16.5 

20.5 

TABLE  C-2 


Mean  Accuracy 

Scores  in  Experiment 

II,  High  Experience 

Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tint 

1. 

Landform  Ident 

16.0 

11.0 

13.5 

2. 

Ridge-Valley  Ident 

32.0 

22.0 

29.0 

3. 

Slope  Ident 

14.0 

8.0 

15.5 

4. 

Ident  of  High-Low  Areas 

49.0 

43.0 

53.5 

5. 

Spot  Elevation 

25.0 

16.5 

24.0 

6. 

Vertical  Profile 

9.0 

6.0 

12.5 

7. 

V isual izat ion 

13.0 

7.0 

10.0 

8. 

Defilade 

22.0 

20.0 

20.5 

I 


|  TABLE  C-3 

!  Analysis  of  Variance  for  Accuracy  Data  Experiment  II, 

Landform  Identification 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

9.38 

2.71 

NS 

r 

r  i 
►* 

Experience 

1 

3.04 

.88 

NS 

* 

‘  i 

A  X  B 

2 

7.35 

2.12 

NS 

w 

Within  cells 

24 

3.46 

J 

Total 

29 

t 

TABLE  04 

Ana  lysis 

of  Variance  for  Accuracv  Data 
Ridge-Valley  Identification 

Experiment  1 1  , 

Source 

df 

MS 

F 

P< 

K  1 

Mao  Format 

2 

64.36 

1.81 

NS 

Experience 

1 

82.03 

2.31 

NS 

A  X  B 

2 

17.07 

.48 

NS 

Within  cells 

24 

35.48 

Tot  a  1 

29 

40 


TABLE  C-5 


Analysis  of  Variance  for  Accuracy  Data  Experiment  II, 

Slope  Identification 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

26,77 

3.19 

NS 

Experience 

1 

2.01 

.24 

NS 

A  X  B 

2 

14.60 

1.74 

NS 

Within  cells 

24 

8.40 

Total 

29 

TABLE  C-6 

Ana lys is 

of  Variance  for  Accuracy  Data 
High-Low  Area  Identification 

Experiment  I I , 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

62.29 

.82 

NS 

Fxper ience 

1 

37.89 

.50 

NS 

A  X  B 

2 

37.62 

.49 

NS 

Within  cells 

24 

76.18 

Total 

29 

41 


TABLE  C-7 


Analysis  of  Variance  for  Accuracy  Data  Experiment  II, 
Spot  Elevation 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

45.07 

1.15 

NS 

Experience 

1 

9.67 

.25 

NS 

A  X  B 

2 

14.16 

.36 

NS 

Within  cells 

24 

39.21 

Total 

29 

TABLE  0-8 

Ana l vs  is 

of  Variance  for  Accuracv  Data  Experiment  11, 
Vertical  Profile  Identification 

TABLE  C-9 


Analysis  of  Variance  for  Accuracy  Data  Experiment  II, 
Terrain  Visualization 


Source 

df 

MS 

F 

P< 

Map  format 

2 

19.45 

2.16 

NS 

Fxper 1 ence 

1 

14.36 

1.59 

NS 

A  X  B 

2 

6.96 

.77 

NS 

Within  cells 

22 

9.01 

Total 

27 

TABLE  C-10 

Analysis 

of  Variance  for  Accuracy 
Defilade 

Data  Experiment  11, 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

9.10 

.75 

NS 

Experience 

1 

15.83 

1.31 

NS 

A  X  B 

2 

4.17 

.35 

NS 

Within  cells 

22 

12.09 

Total 

27 

43  - 


TABLE  C— 1 1 

Summary  of  Significant  Accuracy  Effects  in  Experiment  II 

_ Main  Effects  Interaction 


Subtes  t 

Map  Format 

Experience 

1 . 

Land  form  Ident 

NS 

NS 

NS 

2. 

Ridge-Valley  Ident 

NS 

NS 

NS 

3. 

Slope  Ident 

NS 

NS 

NS 

4. 

Ident  of  High-Low  Areas 

NS 

NS 

NS 

5. 

Spot  Elevation 

NS 

NS 

NS 

6. 

Vertical  Profile 

NS 

NS 

NS 

7. 

V isual izat ion 

NS 

NS 

NS 

8. 

Oef i lade 

NS 

NS 

NS 

APPENDIX  D 


MEDIAN  TIME  EFFECTS  IN  EXPERIMENT  II 
TABLE  D-l 


Median  Time 

Scores  in  Experiment  II 

,  Low  Experience 

Group 

Subtest 

Contour  Lines 

Shaded  Relief 

Layer  Tint 

I, 

Landform  Ident 

.81 

1.15 

.83** 

2. 

Ridge-Valley  Ident 

1.81 

2.27 

1.57 

3. 

Slope  Ident 

1.06 

1.58 

4. 

Ident  of  High-Low  Areas 

2.44 

3.13 

1.99 

5. 

Spot  Elevation3 

Incomplete  data  on  this 

subtes  t 

6. 

Vertical  Profile 

2.39 

1.16 

2.69 

7. 

V  isualizat ion 

.72 

.68 

.92** 

8. 

Def i lade 

2.39 

1.99 

1.93 

a 

Manv  of  the  participants 

were  unable  to  complete 

enough  items  to 

obtain 

meaningful  time  scores  on  this  subtest. 
*p< . 05 

**p< . 0 1 


TABLE  D-2 


Median  Time  Scores 

in  Exper iment  I  1  , 

High  Experience  Croup 

Subtes  t 

Contour  Lines 

Shaded  Relief  Laver  Tint 

1  . 

Landform  Ident 

5.08 

1 .01 

1 .45** 

2. 

Ridge-Valley  Ident 

1 .08 

1.80 

2.5  2 

3. 

Slope  Ident 

.  59 

.94 

1.25 

4. 

Ident  of  High-Low  Areas 

2.83 

1.63 

3.58 

5. 

Spot  Elevation3 

Incomp lete 

data  on  this  subtest 

6. 

Vertical  Profile 

2 . 6  7 

1.17 

/•  03 

7. 

V isua 1 i zat ion 

.87 

.38 

1 . 35** 

8. 

De  f i lade 

1 .06 

1.72 

2.37 

3  Manv  of  the  participants  were  unable  to  complete  enough  items  to  obtain 
meaningful  time  scores  on  this  suhtest. 

**p<\01 
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TABLE  D-3 


Analysis  of  Variance  for  Median  Time,  Experiment  II, 
Landform  Identification 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

5.69 

71.83 

.01 

Experience 

1 

9.59 

121.03 

.01 

A  X  B 

2 

7.05 

89.05 

.01 

Within  cells 

24 

.08 

Total 

29 

Analysis 

TABLF  D-4 

of  Variance  for  Median  Time, 
Ridge-Vallev  Identification 

Experiment  II , 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

.60 

.92 

NS 

Experience 

1 

.03 

.04 

.05 

A  X  B 

2 

1.04 

1.58 

NS 

Within  cells 

24 

.65 

Total 

29 

46 


TABLE  D-S 


Analysis  of  Variance  for  Median  Time,  Experiment  II, 
Slope  Ident i f icat ion 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

.27 

1.21 

NS 

Experience 

1 

.39 

1.72 

NS 

A  X  B 

2 

.22 

.92 

NS 

Within  cells 

24 

.23 

Total 

29 

Analysis 

TABLE 

of  Variance  for 
Hieh-Low  Area 

D-6 

Median  Time, 
Ident i f icat ion 

Experiment  1 1  , 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

.22 

.13 

NS 

Experience 

1 

.10 

.07 

NS 

A  X  R 

2 

3.07 

2.18 

NS 

Within  cells 

24 

1.41 

Total 

29 

-  47  - 
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TABLE  D-7 


Analysis  of  Variance  for  Median  Time,  Experiment  II, 
Vertical  Profile 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

4.95 

2.20 

NS 

Experience 

1 

.61 

.27 

NS 

A  X  B 

2 

1.01 

.45 

NS 

Within  cells 

23 

2.25 

Total 

28 

Analysis 

TABLE 

of  Variance  for 
Terrain 

D-8 

Median  Time, 
Visualization 

Experiment  II, 

Source 

df 

MS 

F 

P< 

Map  Format 

2 

.46 

8.  14 

.01 

Experience 

1 

.03 

.62 

NS 

A  X  B 

2 

.17 

2.99 

NS 

Within  cells 

22 

.06 

Total 

27 

48- 


TABLE  D-9 


Analysis  of  Variance  for  Median  Time,  Experiment  II, 
Defilade 


Source 

df 

MS 

F 

P< 

Map  Format 

2 

.24 

.30 

NS 

Experience 

1 

.56 

.70 

NS 

A  X  B 

2 

.99 

1.24 

NS 

Within  cells 

22 

.80 

Total  27 


TABLE  D-10 

Summary  of  Significant  Time  Effects  in  Experiment  II 


Subtest 

Main 

Map  Format 

Effects 

Experience 

Interact  ion 

1  . 

Land  form  Ident 

.01 

.01 

.01 

2. 

Ridge-Valley  Ident 

NS 

.05 

NS 

3. 

Slope  Ident 

NS 

NS 

NS 

4. 

Ident  of  High-Low  Areas 

NS 

NS 

NS 

5. 

Spot  Elevation 

NS 

NS 

NS 

6. 

Vertical  Profile 

NS 

NS 

NS 

7. 

Visualization 

.01 

NS 

NS 

8. 

Defilade 

NS 

NS 

NS 
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APPENDIX  E 


FIELD  VALIDATION  OF  EIGHT  SUBTESTS  OF  INFORMATION 
EXTRACTION  FROM  MAP  PRODUCTS 


Introduction 


A  secondary  purpose  of  this  research  effort  was  the  development  of 
a  research  methodology  for  work  in  this  problem  area.  Traditionally, 
new  products  are  evaluated  by  opinion  surveys  from  user  groups  or  by 
field  tests.  There  are  problems  with  both  of  these  methods.  Surveys 
are  inexpensive  and  do  provide  data  on  user  acceptance.  However,  if  the 
users  have  not  actually  used  the  map  under  consideration  their  opinions 
have  little  predictive  validity  (Hill,  1974;  Wheaton,  Zavala,  and  Van 
Cott,  1967).  Field  tests  of  map  products  also  present  problems.  Field 
tasks  involving  a  map  are  complex  and  involve  additional  skills  beyond 
simply  extracting  information  from  a  map.  In  addition  a  large  number  of 
variables  are  present  in  field  trials  which  are  expensive  and  time 
consuming  to  control.  If  only  one  terrain  is  used  the  results  may  be 
terrain  specific  and  may  not  generalize  to  other  terrains.  Seasonal 
factors  are  involved,  and  results  may  not  be  the  same  at  different 
times  of  the  year  since  land  navigation  is  easier  in  the  late  fall  and 
winter  when  vegetation  does  not  obscure  terrain  features.  Many  other 
contaminating  factors  exist;  for  example,  early  participants  make  tracks 
through  vegetation  which  influence  the  behavior  of  later  participants. 

While  field  trials  can  produce  valuable  data  when  these  factors  are 
controlled,  it  would  be  desirable  to  develop  an  alternate  method  for 
assessing  the  effectiveness  of  new  map  formats. 

After  extensive  consultations  with  cartographers  at  the  U.S.  Army 
Engineer  Topographic  Laboratory  it  was  determined  that  measuring  the 
speed  and  accuracy  with  which  terrain  information  could  be  extracted  from 
a  map  would  be  a  valuable  addition  to  user  surveys  and  field  tests  as  a 
method  for  evaluating  cartographic  products.  Extensive  discussions  were 
held  with  military  map  users  at  the  U.S.  Army  Infantry  School  to  determine 
what  types  of  information  must  be  extracted  from  maps  in  Army  tasks.  It 
was  found  that  U.S.  Army  users  extract  eight  types  of  information  from 
maps,  and  a  subtest  was  developed  for  each  type.  These  eight  types  are: 
landform  identification;  ridge-valley  identification;  slope  identification; 
identification  of  high  and  low  areas;  spot  elevation;  vertical  profile; 
terrain  visualization;  and  defilade  or  intervisibility.  These  are 
described  in  greater  detail  in  the  Introduction  to  this  report. 


In  the  development  of  behavioral  measures,  validity  is  an  issue  that 
must  be  addressed.  Since  the  eight  subtests  are  based  on  performance  on 
a  map  use  task,  the  test  is  inherently  valid  in  terms  of  construct 
validity  and  content  validity.  If  a  test  contains  a  representative  sample 
of  the  behavior  domain  to  be  tested,  it  has  content  validity  (Anastasi, 


1976,  pp.  133-134). 
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Predictive  validity,  however,  must  be  demonstrated  when  a  classroom 
type  of  test,  such  as  the  eight  subtests,  is  developed.  The  classroom 
test  must  be  compared  to  a  criterion  where  performance  is  measured,  and 
correlation  must  be  demonstrated.  Since  map  use  tasks  in  the  field  are 
complex  and  contain  skills  other  than  map  reading,  it  would  be  unreasonabl 
to  expect  extremely  high  correlations.  However,  performance  in  the 
field  on  a  map  task  should  correlate  with  a  field  criterion.  For 
determining  the  predictive  validity  of  these  subtests,  field  tasks  of 
checkpoint  location  and  target  location  were  chosen. 


Method 


Participants.  A  total  of  16  participants  were  used.  These  partici¬ 
pants  were  a  stratified  random  sample  chosen  to  represent  the  varying 
degrees  of  experience  in  the  Army  population  at  large.  Participants 
were  obtained  from  installations  other  than  where  the  experiment  was 
being  conducted.  No  participants  had  prior  experience  on  the  terrain 
used  in  the  experiment. 

Location.  The  field  test  portion  of  the  experiment  was  conducted  in 
the  Gabilan  Valey  area  of  Hunter-Liggett  Military  Reservation  in  Californi 
The  U.S.  Army  Combat  Development  Experimentation  Command  (CDEC)  was 
conducting  field  research  in  this  location,  and  the  experiment  was 
supported  by  CDEC  as  a  portion  of  their  research  effort.  The  area  is 
approximately  30  percent  flat-to-rolling  terrain  and  70  percent  rolling- 
to-rough  terrain.  The  floor  of  the  Gabilan  Valley  averages  1200  feet 
above  sea  level  and  is  sparsely  vegetated  with  oak  trees  and  shrub.  The 
surrounding  hills  and  mountains  of  the  Coast  Range  are  heavily  vegetated 
with  oak,  chaparral,  and  shrub.  The  area  permits  cross  country  vehicle 
traffic  in  dry  weather.  Almost  all  of  the  rainfall  in  the  region  occurs 
during  the  winter  months,  and  the  experiment  took  place  in  April  and  May 
when  conditions  were  dry. 

Apparatus .  Apparatus  consisted  of:  1/4  ton  trucks;  a  radar-based 
range  measurement  system  (RMS)  which  used  triangulation  in  conjunction 
with  a  DEC-1061  computer  system  to  determine  accurate  position  location 
information;  a  voice  recording  system;  two-way  radios;  the  map  performance 
measures,  standard  Army  contour  maps  of  the  area,  and  various  forms. 

Procedure .  The  experiment  included  of  both  classroom  and  field  tests. 
After  an  introductory  briefing  participants  were  given  the  eight  map  use 
subtests.  These  subtests  were  administered  in  a  large  room  with  widely 
spaced  tables  in  a  single  session.  The  field  portion  of  the  experiment 
was  conducted  over  the  next  five  weeks. 


Participants  were  given  training  in  map  use  for  land  navigation  and 
safety  before  the  field  portion  of  the  experiment.  Numerous  safety 
problems  are  inherent  with  single  participants  navigating  in  relatively 
isolated  terrain:  they  may  become  lost,  injured  in  accidents,  etc. 
Therefore,  for  safety,  participants  were  tested  singly  using  light  1/4 
ton  trucks  with  drivers.  The  participant  acted  as  a  navigator  and 
directed  the  driver.  Drivers  were  trained  to  avoid  giving  cues  and 
restricted  to  speeds  of  less  than  15  miles  per  hour. 

In  the  field  tests,  participants  were  given  two  types  of  land 
navigation  tasks,  checkpoint  identification  and  target  location.  Four 
routes  were  used  with  six  checkpoints  on  each  route  and  an  observation 
post  at  the  end  of  each  route.  Thus,  each  participant  had  to  locate  24 
checkpoints.  Routes  were  approximately  6  to  7  kilometers  long  and  the 
checkpoints  varied  from  1/2  to  2  kilometers  apart.  The  order  of  the 
routes  for  each  participant  was  counterbalanced ,  and  to  avoid  the 
problem  of  tracks  through  vegetation  becoming  a  cue  for  subsequent 
participants,  vehicles  were  driven  through  the  area  between  trials. 

Participants  were  issued  maps,  paper,  and  protractor.  For  each 
trial  or  route  they  were  given  a  map  with  a  set  of  instructions  and  the 
eight-digit  coordinates  of  the  start  point,  and  six  checkpoints.  They 
were  directed  to  make  a  thorough  map  reconnaissance  and  plot  their 
planned  routes.  For  the  checkpoint  identification  task,  they  were  taken 
to  a  known  start  point  and  directed  to  find  the  six  checkpoints  and  the 
observation  post.  When  the  participants  believed  that  they  had  arrived 
at  a  checkpoint  the  position  was  noted.  The  exact  location  was  determined 
with  the  RMS.  The  participants  were  informed  of  their  actual  position 
at  each  checkpoint  so  that  they  would  always  proceed  from  a  known 
position  and  previous  errors  would  not  be  compounded  in  locating  the  next 
checkpoint.  Checkpoint  location  scores  were  determined  by  noting  the 
radial  error  or  the  direct  line  distance  between  reported  location  of 
the  checkpoint  and  the  actual  location.  Direction  of  errors  was  not 
recorded . 

For  the  target  location  task,  participants  were  directed  to  find  an 
observation  post  at  the  end  of  their  checkpoint  identification  route. 

If  they  were  in  error  they  were  directed  to  the  true  location  of  the 
last  checkpoint.  All  participants  were  given  their  location  in  eight¬ 
digit  coordinates,  and  allowed  use  of  binoculars,  compass,  and  protractor 
as  well  as  their  map.  The  targets  were  I2-foot-square  panels  of  wood 
located  at  distances  varying  between  1000  and  6000  meters  from  the 
observation  post.  Radial  errors  between  the  reported  location  and  the 
actual  location  of  targets  were  calculated  with  the  RMS  system. 
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Two  series  of  nine  targets  were  used.  Participants  were  allowed 
to  use  any  procedures  they  preferred  to  locate  targets  in  the  first 
series.  For  the  second  series  of  targets,  participants  were  directed 
to  use  the  procedures  outlined  in  FM  21-26,  the  Army  Field  Manual  on  Map 
Reading,  to  report  target  locations.  Therefore,  two  sets  of  target 
location  scores  were  obtained,  the  first  using  unspecified  procedures 
and  the  second  using  specified  procedures. 


Results 


On  the  three  field  tasks,  the  mean  radial  error  between  actual 
location  and  reported  location  was  determined.  These  scores  were 
correlated  with  the  map  subtest  scores  using  Spearman-Rho  correlations, 
which  are  more  appropriate  than  the  Pearson  Product  Moment  correlation 
considering  the  number  of  participants  involved.  These  correlations  are 
presented  in  Table  E-l. 


TABLE  E-l 

Correlations  of  Map  Performance  Task  with  Field  Criteria 


Subtest 

Target  Location 

Target  Location 

Checkpoint 

Unspeci f ied 

Speci f ied 

Location 

Procedure 

Procedure 

1. 

Land  form  Ident 

-.30 

-.  10 

-.  34 

2. 

Ridge -Valley  Ident 

-.  14 

-.  18 

-.27 

3. 

Slope  Ident 

-.32 

-.  13 

-.20 

4. 

Ident  of  High-Low  Areas 

-.26 

-.49* 

-.30 

5. 

Spot  Elevation 

-.28 

-.47* 

-.22 

6. 

Vertical  Profile 

-.04 

-.20 

+  .33 

7.  Visualization 


-.24 


-.29 


-.63** 


In  the  first  field  task,  checkpoint  location,  all  eight  correlations 
are  negative,  which  was  the  expected  direction,  since  the  field  perfor¬ 
mance  was  measured  by  an  error  score.  In  the  second  field  performance 
task,  target  location  with  unspecified  procedures,  all  correlations  are 
in  the  expected  direction  with  the  exception  of  defilade.  Two  correla¬ 
tions  are  significant  at  the  .05  level:  identification  of  high  and  low 
areas,  and  spot  elevation.  In  the  third  field  task,  target  location 
with  specified  procedures,  all  correlations  were  in  the  expected  direc¬ 
tion  with  the  exception  of  the  vertical  profile.  The  correlation  on 
terrain  visualization  was  significant  at  the  .01  level. 


Discussion 


Compared  to  previous  techniques  for  evaluating  maps,  the  techniques 
used  in  this  research  may  provide  a  valuable  alternative  for  assessing 
the  effectiveness  of  map  products.  The  results  of  the  field  validation, 
sumnarized  in  Table  E-l,  suggest  that  the  subtests  have  predictive 
validity.  Of  the  24  correlation  coefficients  obtained,  all  but  2  were 
in  the  correct  direction,  which  is  negative  because  the  field  performance 
score  was  an  error  score,  correlated  with  the  subtests. 

Only  3  of  the  24  correlations  obtained  were  statistically  significant 
at  conventional  levels  of  confidence ,  .05  and  .01.  However,  a  number  of 
considerations  must  be  kept  in  mind  in  interpreting  the  data.  The 
number  of  participants  (16)  was  rather  low,  due  to  the  expense  of  field 
tests  and  logistical  considerations;  high  correlations  rarely  occur  with 
a  sample  size  of  this  magnitude.  The  field  criteria  also  must  be 
considered.  A  large  number  of  variables  are  always  present  in  the 
field,  and  many  of  these  cannot  be  controlled.  In  addition,  map  use 
tasks  in  the  field  are  complex  and  a  number  of  skills  are  involved  in 
addition  to  extracting  information  from  a  map  product.  In  considering 
the  impact  of  these  factors,  the  fact  that  22  of  24  correlations  were  in 
the  correct  direction  and  that  three  of  the  correlations  were  significant 
supports  the  predictive  validity  of  the  subtests. 
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